Systems and methods for processing a table of information in a document

ABSTRACT

A device may receive document image data that includes an image of a document to be digitized. The device may detect, from the document image data, a table of information that is depicted in the image. The device may determine a data extraction score associated with a table image, wherein the data extraction score is associated with using a data conversion technique to convert the table image to digitized table data. The device may perform, based on the data extraction score not satisfying a threshold, a morphological operation on the table image to generate an enhanced table image that corresponds to an enhanced table of information associated with the table of information. The device may process, using the data conversion technique, the enhanced table image to extract the information from the enhanced table. The device may perform an action associated with the extracted information.

BACKGROUND

Document processing includes converting typed text on paper-based andelectronic documents (e.g., scanned image of a document) into electronicinformation using intelligent character recognition (ICR), opticalcharacter recognition (OCR), manual data entry, and/or the like.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1D are diagrams of an example implementation described herein.

FIG. 2 is a diagram illustrating an example of training a machinelearning model and applying a trained machine learning model to a newobservation.

FIG. 3 is a diagram of an example environment in which systems and/ormethods described herein may be implemented.

FIG. 4 is a diagram of example components of one or more devices of FIG.3.

FIG. 5 is a flow chart of an example process relating to processing atable of information in a document.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description of example implementations refers tothe accompanying drawings. The same reference numbers in differentdrawings may identify the same or similar elements.

Current document processing techniques typically require manuallycopying or scanning documents into an image format, sorting the imageddocuments, and converting the imaged documents into a digital format(e.g., digitizing the documents). Due to copying and/or scanning errors,such document processing techniques may result in inaccurate digitizeddocuments. Furthermore, in many cases, these document processingtechniques are unable to correctly digitize information contained intables of the documents. Inaccuracies may be compounded when the tableis depicted in an unstructured table (e.g., without lines, borders,dividers, and/or the like) and/or the table is poorly copied and/orscanned such that the table appears rotated and/or tilted when imaged,is associated with an amount of noise that makes the table appearunclear, and/or the like. Thus, current document processing techniquesmay waste computing resources (e.g., processing resources, memoryresources, communication resources, and/or the like), networkingresources, human resources, and/or the like associated with manuallyprocessing documents, generating digital forms of the documents withincorrect table information, correcting the incorrect table informationif discovered, and/or like.

Some implementations described herein provide a system that correctlyprocesses a table of information in a document. In some implementations,the system may detect (e.g., using a machine learning model) a table ofinformation that is depicted in an image of document. The system (e.g.,a document management system), may extract a table image of the tableand may determine table image data of the table (e.g., that indicateswhether the table is rotated, tilted, missing borders, associated withan amount of noise, and/or the like in the table image). The system maydetermine (e.g., using an additional machine learning model), based onthe table image data, a data extraction score associated with the tableimage, wherein the data extraction score indicates whether the tableimage may be converted to digitized table data (e.g., without furtherprocessing). The system may perform, based on the data extraction scorenot satisfying a threshold, a morphological operation to the table imageto generate an enhanced table image. The system may process, using adata conversion technique, the enhanced table image to extractinformation from the enhanced table and may perform an action associatedwith the extracted information (e.g., store the extracted information ina data structure with a digitized form of the document).

In this way, the system utilizes machine learning models, morphologicaloperations, and/or the like to correctly extract table information froma copied and/or scanned document. The system enables a user to performdocument processing (e.g., document information extraction) withoutinvolvement of additional users. This conserves computing resources,networking resources, and/or the like that would have otherwise beenconsumed in manually processing documents, generating digital forms ofthe documents with incorrect table information, correcting the incorrecttable information, and/or like.

FIGS. 1A-1D are diagrams of an example 100 associated with processing atable of information in a document. As shown in FIGS. 1A-1D, example 100includes a document storage device 105, a document management system110, and/or a user device 160. In some implementations, the documentstorage device 105, the document management system 110, and/or the userdevice 160 may be connected via a network, such as a wired network(e.g., the Internet or another data network), a wireless network (e.g.,a wireless local area network, a wireless wide area network, a cellularnetwork, and/or the like), and/or the like.

As shown in FIG. 1A, and by reference number 115, the documentmanagement system 110 may obtain document image data. The document imagedata may include an image of a document (e.g., to be digitized into adigital form), such as a contract, a deal structure document, a paymentschedule document, and/or the like. The image of the document mayinclude one or more tables, where each table includes informationorganized by at least one row, column, header, cell, divider, and/or thelike.

In some implementations, the document management system 110 maycommunicate with the document storage device 105 to obtain the documentimage data via a communication link between the document managementsystem 110 and the document storage device 105. In some implementations,the document storage device 105 may store a corpus of document imagedata associated with multiple documents. For example, the corpus mayinclude document image data associated with documents that were scannedusing a document scanner, a digital camera, and/or the like. As anotherexample the corpus may include document image data associated withnon-machine readable digital documents, such as non-machine readableportable document format (PDF) documents, image file documents, and/orthe like. The document storage device 105 may provide document imagedata associated with a document, of the multiple documents, to thedocument management system 110 (e.g., via the communication link).

As further shown in FIG. 1A, and by reference number 120, the documentmanagement system 110 may process the document image data to obtain textdata associated with the image of the document. For example, thedocument management system 110 may use an optical character recognition(OCR) technique to process the image data to identify the text dataassociated with the image of the document. The text data may indicatestrings of characters, such as numbers, words, phrases, sentences,and/or the like, of the document.

As further shown in FIG. 1A, and by reference number 125, the documentmanagement system 110 may identify and/or detect one or more tables(e.g., one or more tables of information) in the image of the document.In some implementations, the document management system 110 may process(e.g., using a table-based objection detection technique) the documentimage data and/or the text data to identify a table in the image of thedocument. For example, the document management system 110 may processthe document image data and/or the text data to identify a portion ofthe image of the document that includes strings of characters, such asnumbers, words, phrases, sentences, and/or the like, organized and/orarranged in a format that includes at least one row, column, header,cell, divider, and/or the like.

In some implementations, the document management system 110 may processthe image data and/or the text data using a machine learning model toidentify and/or detect a table in the image of the document. Forexample, the document management system 110 may use a machine learningmodel that includes one or more neural networks to process the imagedata and/or the text data to identify a table in the image of thedocument. The one or more neural networks may include a neural networktrained to identify configurations of table markings (e.g., lines,dividers, borders, headers, footers, and/or the like); a neural networktrained to identify arrangements of text (e.g., rows, columns, and/orthe like of text); a neural network trained to identify tableidentifiers (e.g., text indicative of a table, such as “table,” “cell,”“row,” “column,” and/or the like); a neural network trained to identifytypes of tables associated with particular types of documents (e.g. apayment schedule table for a contract, a profit and loss table for anaccounting document, and/or the like); and/or the like.

The machine learning model may have been trained based on training datathat includes images of documents, configurations of tables in theimages, identifications of the tables in the images (e.g., based ontable markings associated with the tables, text columns associated withthe tables, text rows associated with the tables, table identifiersassociated with the tables, and/or the like), and/or the like. Thedocuments may include one or more sets of documents, where each set ofdocuments includes documents that are the same type of document, thathave a same type of table (e.g., in terms of information content, tablestructure, table formatting, and/or the like); that are associated withthe same entity (e.g., the same individual, organization, company,and/or the like); and/or the like. Using the training data as input tothe machine learning model, the machine learning model may be trained toidentify one or more relationships in the training data (e.g., betweenthe images, the configurations of the tables in the images, theidentifications of the tables, and/or the like) to identify and/ordetect tables in various images of documents. The machine learning modelmay be trained and/or used in a similar manner to that described belowwith respect to FIG. 2.

In some implementations, the document management system 110 may verifythat the table includes a single table. For example, the documentmanagement system 110 may verify that the table includes a single tablebased on header information of the table, a table identifier of thetable, table markings of the table (e.g., that are indicated by thedocument image data and/or the text data). When the document managementsystem 110 determines that the table is not a single table (e.g., thetable comprises one or more separate tables), the document managementsystem 110 may process image data and/or the text data associated withindividual portions of the image of the document to identify eachseparate table (e.g., in a similar manner as that described above).Alternatively, when the document management system 110 verifies that thetable includes a single table, the document management system 110 mayperform one or more operations described herein in relation to FIGS.1B-1D.

As shown in FIG. 1B, and by reference number 130, the documentmanagement system 110 may extract a table image that depicts the tablefrom the image of the document (e.g., that depicts the portion of theimage of the document that includes the table). For example, thedocument management system 110 may extract the table image by croppingthe image of the document to include only the portion of the image ofthe document that includes the table.

As further shown in FIG. 1B, and by reference number 135, the documentmanagement system 110 may analyze the table image to identify tableimage data associated with the table from the table image. The tableimage data may indicate configurations of table markings (e.g., lines,dividers, borders, headers, footers, and/or the like) of the tableimage, arrangements of text (e.g., rows, columns, and/or the like, oftext) of the table image, table identifiers (e.g., text indicative of atable, such as “table,” “cell,” “row,” “column,” and/or the like) of thetable image, and/or the like. For example, the table image data mayindicate a rotational angle of the table in the table image (e.g., thatthe table is rotated 90 degrees counter-clockwise in the table image, asshown in FIG. 1B). As another example, the table image data may indicatea tilt angle of the table in the table image (e.g., an angle betweenlines associated with rows of the table in the table image and the linesassociated with columns of the table in the table image, as shown inFIG. 1B). In another example, the table image data may indicate whetherthe table is missing any borders in the table image (e.g., between rows,columns, cells, headers, footers, and/or the like of the table in thetable image). In an additional example, the table image data mayindicate an amount of noise associated with the table image (e.g., thataffects a legibility and/or readability of information included in thetable). The table image data may be associated with one or more issues,such as a rotation issue (e.g., the table is rotated more than anthreshold number of degrees in the table image), a tilt angle issue(e.g., the table is tilted more than a threshold number of degrees inthe table image), a missing border issue (e.g., between rows, columns,headers, footers, and/or the like of the table in the table image), atable image noise issue (e.g., an amount of noise associated with thetable image is greater than a noise threshold), and/or the like.

As further shown in FIG. 1B, and by reference number 140, the documentmanagement system 110 may determine a data extraction score associatedwith the table image (e.g., based on the table image data). The dataextraction score may indicate a likelihood that the table image cansuccessfully be digitized in its current form. For example, the dataextraction score may indicate whether the table image can be convertedto digitized table data using a data conversion technique, such as anOCR technique (e.g., without performing a morphological operation, asdescribed herein in relation to FIG. 1C and reference number 145).

In some implementations, the document management system 110 may processthe table image data using an additional machine learning model todetermine the data extraction score. The additional machine learningmodel may have been trained based on training data that includes tableimage data (e.g., associated with table images), determinations of dataextraction scores associated with the historical table data, and/or thelike. Using the historical data as input to the additional machinelearning model, the additional machine learning model may be trained toidentify one or more relationships in the training data (e.g., betweenthe table data, the determinations of data extraction scores, and/or thelike) to determine data extraction scores for various table image data.The additional machine learning model may be trained and/or used in asimilar manner to that described below with respect to FIG. 2.

In some implementations, the document management system 110 maydetermine whether the data extraction score satisfies (e.g., is greaterthan or equal to) a threshold (e.g., a threshold associated withsuccessfully converting the table image to digitized table data usingthe data conversion technique). When the document management system 110determines that the extraction score satisfies the threshold, thedocument management system 110 may process the table image using thedata conversion technique (e.g., in a similar manner as that describedherein in relation to FIG. 1D and reference number 150).

As shown in FIG. 1C, and by reference number 145, when the documentmanagement system 110 determines that the extraction score does notsatisfy the threshold, the document management system 110 may determinethat the table image is to be enhanced. In some implementations, thedocument management system 110 may perform one or more morphologicaloperations to the table image to generate an enhanced table image. Theone or more morphological operations may comprise a rotation of thetable in the table image (e.g., to reduce or remove a rotational angleof the table in the table image); a reduction to a tilt of the tableimage (e.g., to reduce or remove a tilt angle of the table in the tableimage); an addition and/or enhancement of borders, lines, dividers,and/or the like, of the table image; a reduction to noise of the tableimage (e.g., to reduce noise associated with the table in the tableimage); and/or the like.

For example, to rotate the table in the table image, the documentmanagement system 110 may determine (e.g., based on the table imagedata) the rotational angle of the table in the table image and cause thetable to be rotated in an opposite direction equal to the rotationalangle (e.g., cause the table to be rotated −θ° when the rotational angleis θ°) in the enhanced table image. The document management system 110may rotate the table in the table image when the rotational anglesatisfies (e.g., is greater than or equal) to a threshold rotationalangle.

As another example, to reduce the tilt of the table image, the documentmanagement system 110 may determine (e.g., based on the table imagedata) the tilt angle of the table in the table image and cause the tableto be tilted in an opposite direction equal to the tilt angle (e.g.,cause the table to be tilted −θ° when the tilt angle is θ°) in theenhanced table image. The document management system 110 may tilt thetable in the table in the table image when the tilt angle satisfies(e.g., is greater than or equal) to a threshold tilt angle.

In an additional example, to add and/or enhance borders, lines,dividers, and/or the like, of the table image, the document managementsystem 110 may identify vertical and/or horizontal division lines (e.g.,composed of white space) associated with the table image and maygenerate (e.g., based on the vertical and/or horizontal division lines)borders, lines, dividers, and/or the like to add to the table image andthereby depict each cell, header, footer, and/or the like of the tableas enclosed in the enhanced table image. The document management system110 may add and/or enhance borders, lines dividers, and/or the like, ofthe table image the table image when the table image data indicates thata threshold percentage of the table depicted in the table image ismissing borders, lines, dividers, and/or the like.

In another example, to reduce noise associated with the table image, thedocument management system 110 may apply a noise reduction imageprocessing technique (e.g., a smoothing processing technique, a medianfiltering processing technique, and/or the like) to the table image togenerate the enhanced table image. The document management system 110may apply the noise reduction image processing technique to the tableimage when the amount of noise associated with the table image satisfies(e.g., is greater than or equal to) a noise threshold.

As shown in FIG. 1D, and by reference number 150, the documentmanagement system 110 may process the enhanced table image to extracttable information from the enhanced table image (e.g., information thatindicates the text of each header, each footer, each cell, and/or thelike of the table depicted in the enhanced table image). In someimplementations, the document management system 110 may process theenhanced table image using a data conversion technique, such as an OCRtechnique, to extract the table information from the enhanced table. Insome implementations, the document management system 110 may perform aposition-based extraction on the enhanced table image to identify fields(e.g., cells, headers, footers, and/or the like) of the table in theenhanced table image based on table markings of the enhanced table image(e.g., that indicate respective positions of the fields of the table).Additionally, or alternatively, the document management system 110 mayperform text-based extraction on the enhanced table to identify textwithin the fields of the table in the enhanced table image.

As shown in FIG. 1D, and by reference number 155, the documentmanagement system 110 may perform one or more actions associated withthe extracted table information.

The one or more actions may include the document management system 110providing the extracted table information to a user interface and/or toa user device 160. For example, the document management system 110 mayautomatically provide the extracted table information to usersassociated with processing the image of the document, to end-users(e.g., vendors, customers, applicants, providers, and/or the like) ofprocesses associated with the document, and/or the like. In this way,the document management system 110 may enable a user or an end-userassociated with the user device 160 to consume, apply, process, confirm,and/or the like the extracted table information. For example, if theextracted table information is associated a payment schedule of acontract, a party to the contract may utilize the extracted tableinformation to determine when and/or how payments are to be madeaccording to the payment schedule. This may conserve computingresources, networking resources, human resources, and/or the like thatwould have otherwise been consumed to manually search for the paymentschedule in the contract, digitize the payment schedule, correct thedigitized payment schedule, and/or provide the digitized paymentschedule.

The one or more actions may include the document management system 110storing the extracted table information in a data structure (e.g., inassociation with a digitized version of the document). For example, theprocessing platform may store the extracted table information (e.g., inan entry associated with the digitized version of the document) in adata structure, such as a database, a table, a list, and/or the like,associated with the document management system 110. The user device 160may access the extracted table information from the data structure anddisplay the extracted table information, portions of the extracted tableinformation, the digitized document, and/or the like. In this way, auser may quickly and/or easily obtain the extracted table information,the digitized document, and/or the like.

The one or more actions may include integrating the extracted tableinformation into related data associated with related documents that areassociated with the document. For example, the document managementsystem 110 may identify related data associated with the extracted tableinformation (e.g., related data, such as other extracted tableinformation, that is associated with related documents that areassociated with the document, such as amendments to documents that areassociated with a contract). The document management system 110 mayintegrate the extracted table information with the related data (e.g.,link the extracted table information to the related data in the datastructure of the document management system 110). In this way, a usermay quickly and/or easily obtain the extracted table information, therelated data, and/or the like.

The one or more actions may include retraining, based on feedbackassociated with the extracted table information, the machine learningmodel. For example, a user of the user device 160 may provide feedback(e.g., regarding the quality of the extracted table information, theaccuracy of the extracted table information, and/or the like) to thedocument management system 110 and the document management system 110may retrain the machine learning model based on the feedback. In thisway, the document management system 110 may improve the accuracy of themachine learning model, which may improve speed and efficiency of themachine learning model and conserve computing resources, networkresources, and/or the like.

As indicated above, FIGS. 1A-1D are provided as an example. Otherexamples may differ from what is described with regard to FIGS. 1A-1D.The number and arrangement of devices shown in FIGS. 1A-1D are providedas an example. In practice, there may be additional devices, fewerdevices, different devices, or differently arranged devices than thoseshown in FIGS. 1A-1D. Furthermore, two or more devices shown in FIGS.1A-1D may be implemented within a single device, or a single deviceshown in FIGS. 1A-1D may be implemented as multiple, distributeddevices. Additionally, or alternatively, a set of devices (e.g., one ormore devices) shown in FIGS. 1A-1D may perform one or more functionsdescribed as being performed by another set of devices shown in FIGS.1A-1D.

FIG. 2 is a diagram illustrating an example 200 of training and using amachine learning model in connection with determining a data extractionscore. The machine learning model training and usage described hereinmay be performed using a machine learning system. The machine learningsystem may include or may be included in a computing device, a server, acloud computing environment, and/or the like, such as the documentmanagement system 110 described in more detail elsewhere herein.

As shown by reference number 205, a machine learning model may betrained using a set of observations. The set of observations may beobtained from historical data, such as data gathered during one or moreprocesses described herein. In some implementations, the machinelearning system may receive the set of observations (e.g., as input)from the document storage device 105, the document management system110, and/or the like as described elsewhere herein.

As shown by reference number 210, the set of observations includes afeature set. The feature set may include a set of variables, and avariable may be referred to as a feature. A specific observation mayinclude a set of variable values (or feature values) corresponding tothe set of variables. In some implementations, the machine learningsystem may determine variables for a set of observations and/or variablevalues for a specific observation based on input received from thedocument storage device 105, the document management system 110, and/orthe like. For example, the machine learning system may identify afeature set (e.g., one or more features and/or feature values) byextracting the feature set from structured data, by performing naturallanguage processing to extract the feature set from unstructured data,by receiving input from an operator, and/or the like.

As an example, a feature set for a set of observations may include afirst feature of a tilt angle (e.g., of a table in a table image), asecond feature of border metric (e.g., indicating a percentage, or othermetric, of missing borders of a table in a table image), a third featureof a noisiness metric (e.g., indicating a percentage, or other metric,of noise associated with the table image), and so on. As shown, for afirst observation, the first feature may have a value of 11°, the secondfeature may have a value of 3%, the third feature may have a value of10%, and so on. These features and feature values are provided asexamples, and may differ in other examples.

As shown by reference number 215, the set of observations may beassociated with a target variable. The target variable may represent avariable having a numeric value, may represent a variable having anumeric value that falls within a range of values or has some discretepossible values, may represent a variable that is selectable from one ofmultiple options (e.g., one of multiples classes, classifications,labels, and/or the like), may represent a variable having a Booleanvalue, and/or the like. A target variable may be associated with atarget variable value, and a target variable value may be specific to anobservation. In example 200, the target variable is a data extractionscore, which has a value of 0.15 for the first observation.

The target variable may represent a value that a machine learning modelis being trained to predict, and the feature set may represent thevariables that are input to a trained machine learning model to predicta value for the target variable. The set of observations may includetarget variable values so that the machine learning model can be trainedto recognize patterns in the feature set that lead to a target variablevalue. A machine learning model that is trained to predict a targetvariable value may be referred to as a supervised learning model.

In some implementations, the machine learning model may be trained on aset of observations that do not include a target variable. This may bereferred to as an unsupervised learning model. In this case, the machinelearning model may learn patterns from the set of observations withoutlabeling or supervision, and may provide output that indicates suchpatterns, such as by using clustering and/or association to identifyrelated groups of items within the set of observations.

As shown by reference number 220, the machine learning system may traina machine learning model using the set of observations and using one ormore machine learning algorithms, such as a regression algorithm, adecision tree algorithm, a neural network algorithm, a k-nearestneighbor algorithm, a support vector machine algorithm, and/or the like.After training, the machine learning system may store the machinelearning model as a trained machine learning model 225 to be used toanalyze new observations.

As shown by reference number 230, the machine learning system may applythe trained machine learning model 225 to a new observation, such as byreceiving a new observation and inputting the new observation to thetrained machine learning model 225. As shown, the new observation mayinclude a first feature of 15°, a second feature of 11%, a third featureof 26%, and so on, as an example. The machine learning system may applythe trained machine learning model 225 to the new observation togenerate an output (e.g., a result). The type of output may depend onthe type of machine learning model and/or the type of machine learningtask being performed. For example, the output may include a predictedvalue of a target variable, such as when supervised learning isemployed. Additionally, or alternatively, the output may includeinformation that identifies a cluster to which the new observationbelongs, information that indicates a degree of similarity between thenew observation and one or more other observations, and/or the like,such as when unsupervised learning is employed.

As an example, the trained machine learning model 225 may predict avalue of 0.46 for the target variable of the data extraction score forthe new observation, as shown by reference number 235. Based on thisprediction, the machine learning system may provide a firstrecommendation, may provide output for determination of a firstrecommendation, may perform a first automated action, may cause a firstautomated action to be performed (e.g., by instructing another device toperform the automated action), and/or the like. The first recommendationmay include, for example, performing one or more morphologicaloperations to enhance the table image. The first automated action mayinclude, for example, automatically performing the one or moremorphological operations to enhance the table image.

In some implementations, the trained machine learning model 225 mayclassify (e.g., cluster) the new observation in a cluster, as shown byreference number 240. The observations within a cluster may have athreshold degree of similarity. As an example, if the machine learningsystem classifies the new observation in a first cluster (e.g., one ormore morphological operations needed), then the machine learning systemmay provide a first recommendation, such as the first recommendationdescribed above. Additionally, or alternatively, the machine learningsystem may perform a first automated action and/or may cause a firstautomated action to be performed (e.g., by instructing another device toperform the automated action) based on classifying the new observationin the first cluster, such as the first automated action describedabove.

In some implementations, the recommendation and/or the automated actionassociated with the new observation may be based on a target variablevalue having a particular label (e.g., classification, categorization,and/or the like), may be based on whether a target variable valuesatisfies one or more threshold (e.g., whether the target variable valueis greater than a threshold, is less than a threshold, is equal to athreshold, falls within a range of threshold values, and/or the like),may be based on a cluster in which the new observation is classified,and/or the like.

In this way, the machine learning system may apply a rigorous andautomated process to determining a data extraction score. The machinelearning system enables recognition and/or identification of tens,hundreds, thousands, or millions of features and/or feature values fortens, hundreds, thousands, or millions of observations, therebyincreasing accuracy and consistency and reducing delay associated withdetermining the data extraction score relative to requiring computingresources to be allocated for tens, hundreds, or thousands of operatorsto manually determining the data extraction score using the features orfeature values.

As indicated above, FIG. 2 is provided as an example. Other examples maydiffer from what is described in connection with FIG. 2.

FIG. 3 is a diagram of an example environment 300 in which systemsand/or methods described herein may be implemented. As shown in FIG. 3,environment 300 may include a document management system 110, which mayinclude one or more elements of and/or may execute within a cloudcomputing system 302. The cloud computing system 302 may include one ormore elements 303-313, as described in more detail below. As furthershown in FIG. 3, environment 300 may include a network 320, a userdevice 160, and/or a document storage device 105. Devices and/orelements of environment 300 may interconnect via wired connectionsand/or wireless connections.

The cloud computing system 302 includes computing hardware 303, aresource management component 304, a host operating system (OS) 305,and/or one or more virtual computing systems 306. The resourcemanagement component 304 may perform virtualization (e.g., abstraction)of computing hardware 303 to create the one or more virtual computingsystems 306. Using virtualization, the resource management component 304enables a single computing device (e.g., a computer, a server, and/orthe like) to operate like multiple computing devices, such as bycreating multiple isolated virtual computing systems 306 from computinghardware 303 of the single computing device. In this way, computinghardware 303 can operate more efficiently, with lower power consumption,higher reliability, higher availability, higher utilization, greaterflexibility, and lower cost than using separate computing devices.

Computing hardware 303 includes hardware and corresponding resourcesfrom one or more computing devices. For example, computing hardware 303may include hardware from a single computing device (e.g., a singleserver) or from multiple computing devices (e.g., multiple servers),such as multiple computing devices in one or more data centers. Asshown, computing hardware 303 may include one or more processors 307,one or more memories 308, one or more storage components 309, and/or oneor more networking components 310. Examples of a processor, a memory, astorage component, and a networking component (e.g., a communicationcomponent) are described elsewhere herein.

The resource management component 304 includes a virtualizationapplication (e.g., executing on hardware, such as computing hardware303) capable of virtualizing computing hardware 303 to start, stop,and/or manage one or more virtual computing systems 306. For example,the resource management component 304 may include a hypervisor (e.g., abare-metal or Type 1 hypervisor, a hosted or Type 2 hypervisor, and/orthe like) or a virtual machine monitor, such as when the virtualcomputing systems 306 are virtual machines 311. Additionally, oralternatively, the resource management component 304 may include acontainer manager, such as when the virtual computing systems 306 arecontainers 312. In some implementations, the resource managementcomponent 304 executes within and/or in coordination with a hostoperating system 305.

A virtual computing system 306 includes a virtual environment thatenables cloud-based execution of operations and/or processes describedherein using computing hardware 303. As shown, a virtual computingsystem 306 may include a virtual machine 311, a container 312, a hybridenvironment 313 that includes a virtual machine and a container, and/orthe like. A virtual computing system 306 may execute one or moreapplications using a file system that includes binary files, softwarelibraries, and/or other resources required to execute applications on aguest operating system (e.g., within the virtual computing system 306)or the host operating system 305.

Although the document management system 110 may include one or moreelements 303-313 of the cloud computing system 302, may execute withinthe cloud computing system 302, and/or may be hosted within the cloudcomputing system 302, in some implementations, the document managementsystem 110 may not be cloud-based (e.g., may be implemented outside of acloud computing system) or may be partially cloud-based. For example,the document management system 110 may include one or more devices thatare not part of the cloud computing system 302, such as device 400 ofFIG. 4, which may include a standalone server or another type ofcomputing device. The document management system 110 may perform one ormore operations and/or processes described in more detail elsewhereherein.

Network 320 includes one or more wired and/or wireless networks. Forexample, network 320 may include a cellular network, a public landmobile network (PLMN), a local area network (LAN), a wide area network(WAN), a private network, the Internet, and/or the like, and/or acombination of these or other types of networks. The network 320 enablescommunication among the devices of environment 300.

The user device 160 includes one or more devices capable of receiving,generating, storing, processing, and/or providing information, asdescribed elsewhere herein. The user device 160 may include acommunication device and/or a computing device. For example, the userdevice 160 may include a wireless communication device, a user equipment(UE), a mobile phone (e.g., a smart phone or a cell phone, among otherexamples), a laptop computer, a tablet computer, a handheld computer, adesktop computer, a gaming device, a wearable communication device(e.g., a smart wristwatch or a pair of smart eyeglasses, among otherexamples), an Internet of Things (IoT) device, or a similar type ofdevice. The user device 160 may communicate with one or more otherdevices of environment 300, as described elsewhere herein. In someimplementations, the user device 160 may display, to a user of userdevice 160, extracted table information provided by the documentmanagement system 110.

The document storage device 105 includes one or more devices capable ofreceiving, generating, storing, processing, providing, and/or routinginformation, as described elsewhere herein. The document storage device105 may include a communication device and/or a computing device. Forexample, the document storage device 105 may include a server, anapplication server, a client server, a web server, a database server, ahost server, a proxy server, a virtual server (e.g., executing oncomputing hardware), a server in a cloud computing system, a device thatincludes computing hardware used in a cloud computing environment, or asimilar type of device. The document storage device 105 may communicatewith one or more other devices of environment 300, as describedelsewhere herein. In some implementations, the document storage device105 may store extracted table information provided by the documentmanagement system 110.

The number and arrangement of devices and networks shown in FIG. 3 areprovided as an example. In practice, there may be additional devicesand/or networks, fewer devices and/or networks, different devices and/ornetworks, or differently arranged devices and/or networks than thoseshown in FIG. 3. Furthermore, two or more devices shown in FIG. 3 may beimplemented within a single device, or a single device shown in FIG. 3may be implemented as multiple, distributed devices. Additionally, oralternatively, a set of devices (e.g., one or more devices) ofenvironment 300 may perform one or more functions described as beingperformed by another set of devices of environment 300.

FIG. 4 is a diagram of example components of a device 400, which maycorrespond to document management system 110, computing hardware 303,user device 160, and/or document storage device 105. In someimplementations, document management system 110, computing hardware 303,user device 160, and/or document storage device 105 may include one ormore devices 400 and/or one or more components of device 400. As shownin FIG. 4, device 400 may include a bus 410, a processor 420, a memory430, a storage component 440, an input component 450, an outputcomponent 460, and a communication component 470.

Bus 410 includes a component that enables wired and/or wirelesscommunication among the components of device 400. Processor 420 includesa central processing unit, a graphics processing unit, a microprocessor,a controller, a microcontroller, a digital signal processor, afield-programmable gate array, an application-specific integratedcircuit, and/or another type of processing component. Processor 420 isimplemented in hardware, firmware, or a combination of hardware andsoftware. In some implementations, processor 420 includes one or moreprocessors capable of being programmed to perform a function. Memory 430includes a random access memory, a read only memory, and/or another typeof memory (e.g., a flash memory, a magnetic memory, and/or an opticalmemory).

Storage component 440 stores information and/or software related to theoperation of device 400. For example, storage component 440 may includea hard disk drive, a magnetic disk drive, an optical disk drive, a solidstate disk drive, a compact disc, a digital versatile disc, and/oranother type of non-transitory computer-readable medium. Input component450 enables device 400 to receive input, such as user input and/orsensed inputs. For example, input component 450 may include a touchscreen, a keyboard, a keypad, a mouse, a button, a microphone, a switch,a sensor, a global positioning system component, an accelerometer, agyroscope, an actuator, and/or the like. Output component 460 enablesdevice 400 to provide output, such as via a display, a speaker, and/orone or more light-emitting diodes. Communication component 470 enablesdevice 400 to communicate with other devices, such as via a wiredconnection and/or a wireless connection. For example, communicationcomponent 470 may include a receiver, a transmitter, a transceiver, amodem, a network interface card, an antenna, and/or the like.

Device 400 may perform one or more processes described herein. Forexample, a non-transitory computer-readable medium (e.g., memory 430and/or storage component 440) may store a set of instructions (e.g., oneor more instructions, code, software code, program code, and/or thelike) for execution by processor 420. Processor 420 may execute the setof instructions to perform one or more processes described herein. Insome implementations, execution of the set of instructions, by one ormore processors 420, causes the one or more processors 420 and/or thedevice 400 to perform one or more processes described herein. In someimplementations, hardwired circuitry may be used instead of or incombination with the instructions to perform one or more processesdescribed herein. Thus, implementations described herein are not limitedto any specific combination of hardware circuitry and software.

The number and arrangement of components shown in FIG. 4 are provided asan example. Device 400 may include additional components, fewercomponents, different components, or differently arranged componentsthan those shown in FIG. 4. Additionally, or alternatively, a set ofcomponents (e.g., one or more components) of device 400 may perform oneor more functions described as being performed by another set ofcomponents of device 400.

FIG. 5 is a flowchart of an example process 500 associated withprocessing a table of information in a document. In someimplementations, one or more process blocks of FIG. 5 may be performedby a device (e.g., document management system 110). In someimplementations, one or more process blocks of FIG. 5 may be performedby another device or a group of devices separate from or including thedevice, such as document storage device 105, user device 160, computinghardware 303, and/or the like. Additionally, or alternatively, one ormore process blocks of FIG. 5 may be performed by one or more componentsof device 400, such as processor 420, memory 430, storage component 440,input component 450, output component 460, and/or communicationcomponent 470.

As shown in FIG. 5, process 500 may include receiving document imagedata that includes an image of a document to be digitized (block 510).For example, the device may receive document image data that includes animage of a document to be digitized, as described above.

As further shown in FIG. 5, process 500 may include detecting, from thedocument image data, a table of information that is depicted in theimage (block 520). For example, the device may detect, from the documentimage data, a table of information that is depicted in the image, asdescribed above. In some implementations, the table of information isdetected using a machine learning model that is trained to detect tablesbased on at least one of table markings associated with historicaltables depicted in historical images of historical documents, textcolumns associated with the historical tables, text rows associated withthe historical tables, or table identifiers associated with thehistorical tables. The historical documents are associated with thedocument based on the historical documents and the document involvingleast one of a same type of document, a same type of table, or a sameindividual or organization.

In some implementations, the machine learning model comprises at leastone of a neural network trained to identify configurations of tablemarkings, a neural network trained to identify arrangements of text, aneural network trained to identify table identifiers, or a neuralnetwork trained to identify types of tables associated with thedocument.

As further shown in FIG. 5, process 500 may include analyzing a tableimage that is associated with the table of information, to extract tabledata, associated with the table of information, from the table image(block 530). For example, the device may analyze a table image that isassociated with the table of information, to extract table data,associated with the table of information, from the table image, asdescribed above. In some implementations, process 500 includes prior toanalyzing the table image, verifying that the table of informationincludes a single table based on at least one of header information ofthe table of information, a table identifier of the table ofinformation, or table markings of the table of information. In someimplementations, process 500 includes prior to analyzing the tableimage, extracting the table image from an image of a document thatdepicts the table of information, wherein the table of information wasidentified using a machine learning model that is trained to detecttables.

As further shown in FIG. 5, process 500 may include determining a dataextraction score associated with the table image (block 540). Forexample, the device may determine a data extraction score associatedwith the table image, as described above. In some implementations, thedata extraction score is associated with using a data conversiontechnique to convert the table image to digitized table data, isrepresentative of a probability that the data conversion technique canbe used to convert the table image to digitized table data, and/or thelike. In some implementations, the data extraction score is determinedbased on at least one of a rotational position of the table, a tiltangle of the table, or an amount of noise associated with the table.

As further shown in FIG. 5, process 500 may include performing, based onthe data extraction score not satisfying a threshold, a morphologicaloperation to the table image to generate an enhanced table image thatcorresponds to an enhanced table of information associated with thetable of information (block 550). For example, the device may perform,based on the data extraction score not satisfying a threshold, amorphological operation to the table image to generate an enhanced tableimage that corresponds to an enhanced table of information associatedwith the table of information, as described above. In someimplementations, process 500 includes prior to performing themorphological operation, determining that the data extraction score isassociated with an issue that prevents the data conversion techniquefrom being able to convert the table image to digitized table data, andselecting, from a plurality of morphological operations, themorphological operation based on the issue.

In some implementations, the morphological operation comprises at leastone of a rotation of the table image, a reduction to a tilt of thetable, or a reduction of an amount of noise associated with the table.In some implementations, performing the morphological operation includesdetermining, based on an analysis of the table image, an issueassociated with using the data conversion technique to extract theinformation from the table, selecting, from a plurality of morphologicaloperations, the morphological operation that is associated with theissue, and applying the morphological operation to the table image. Theissue comprises at least one of the table being rotated in the tableimage relative to other text of the document, the table being tilted inthe table image, table markings of the table being unclear, or text ofthe table being unclear.

As further shown in FIG. 5, process 500 may include processing, usingthe data conversion technique, the enhanced table image to extract theinformation from the enhanced table (block 560). For example, the devicemay process, using the data conversion technique, the enhanced tableimage to extract the information from the enhanced table, as describedabove. In some implementations, process 500 includes performing aposition-based extraction on the enhanced table image to identify fieldsof the table in the enhanced table image based on table markings of theenhanced table image, or performing text-based extraction on theenhanced table to identify text within the fields of the table in theenhanced table image.

As further shown in FIG. 5, process 500 may include performing an actionassociated with the extracted information (block 570). For example, thedevice may perform an action associated with the extracted information,as described above. In some implementations, performing the actioncomprises at least one of providing, via a display of a user interface,the information, storing the information in a data structure inassociation with a digitized version of the document, integrating theinformation into related data associated with related documents that areassociated with the document, or retraining, based on feedbackassociated with the information, a machine learning model associatedwith identifying the table and performing the morphological operation.Storing the extracted information as digitized table data includes atleast one of identifying related data associated with the extractedinformation, wherein the related data is associated with relateddocuments that are associated with the document, and integrating theinformation with the related data associated with the related documentsthat are associated with the document.

Although FIG. 5 shows example blocks of process 500, in someimplementations, process 500 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 5. Additionally, or alternatively, two or more of theblocks of process 500 may be performed in parallel.

The foregoing disclosure provides illustration and description, but isnot intended to be exhaustive or to limit the implementations to theprecise form disclosed. Modifications and variations may be made inlight of the above disclosure or may be acquired from practice of theimplementations.

As used herein, the term “component” is intended to be broadly construedas hardware, firmware, or a combination of hardware and software.

As used herein, satisfying a threshold may, depending on the context,refer to a value being greater than the threshold, more than thethreshold, higher than the threshold, greater than or equal to thethreshold, less than the threshold, fewer than the threshold, lower thanthe threshold, less than or equal to the threshold, equal to thethreshold, etc., depending on the context.

Certain user interfaces have been described herein. A user interface mayinclude a graphical user interface, a non-graphical user interface, atext-based user interface, and/or the like. A user interface may provideinformation for display. In some implementations, a user may interactwith the information, such as by providing input via an input componentof a device that provides the user interface for display. In someimplementations, a user interface may be configurable by a device and/ora user (e.g., a user may change the size of the user interface,information provided via the user interface, a position of informationprovided via the user interface, etc.). Additionally, or alternatively,a user interface may be pre-configured to a standard configuration, aspecific configuration based on a type of device on which the userinterface is displayed, and/or a set of configurations based oncapabilities and/or specifications associated with a device on which theuser interface is displayed.

To the extent the aforementioned implementations collect, store, oremploy personal information of individuals, it should be understood thatsuch information shall be used in accordance with all applicable lawsconcerning protection of personal information. Additionally, thecollection, storage, and use of such information can be subject toconsent of the individual to such activity, for example, through wellknown “opt-in” or “opt-out” processes as can be appropriate for thesituation and type of information. Storage and use of personalinformation can be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

It will be apparent that systems and/or methods described herein may beimplemented in different forms of hardware, firmware, and/or acombination of hardware and software. The actual specialized controlhardware or software code used to implement these systems and/or methodsis not limiting of the implementations. Thus, the operation and behaviorof the systems and/or methods are described herein without reference tospecific software code—it being understood that software and hardwarecan be used to implement the systems and/or methods based on thedescription herein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of various implementations. In fact,many of these features may be combined in ways not specifically recitedin the claims and/or disclosed in the specification. Although eachdependent claim listed below may directly depend on only one claim, thedisclosure of various implementations includes each dependent claim incombination with every other claim in the claim set.

No element, act, or instruction used herein should be construed ascritical or essential unless explicitly described as such. Also, as usedherein, the articles “a” and “an” are intended to include one or moreitems, and may be used interchangeably with “one or more.” Further, asused herein, the article “the” is intended to include one or more itemsreferenced in connection with the article “the” and may be usedinterchangeably with “the one or more.” Furthermore, as used herein, theterm “set” is intended to include one or more items (e.g., relateditems, unrelated items, a combination of related and unrelated items,etc.), and may be used interchangeably with “one or more.” Where onlyone item is intended, the phrase “only one” or similar language is used.Also, as used herein, the terms “has,” “have,” “having,” or the like areintended to be open-ended terms. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise. Also, as used herein, the term “or” is intended to beinclusive when used in a series and may be used interchangeably with“and/or,” unless explicitly stated otherwise (e.g., if used incombination with “either” or “only one of”).

What is claimed is:
 1. A method, comprising: receiving, by a device,document image data that includes an image of a document to bedigitized; detecting, by the device and from the document image data, atable of information that is depicted in the image; analyzing, by thedevice, a table image that is associated with the table of information,to extract table data, associated with the table of information, fromthe table image; determining, by the device, a data extraction scoreassociated with the table image, wherein the data extraction score isassociated with using a data conversion technique to convert the tableimage to digitized table data; performing, by the device and based onthe data extraction score not satisfying a threshold, a morphologicaloperation to the table image to generate an enhanced table image thatcorresponds to an enhanced table of information associated with thetable of information; processing, by the device and using the dataconversion technique, the enhanced table image to extract theinformation from the enhanced table; and performing, by the device, anaction associated with the extracted information.
 2. The method of claim1, wherein the table of information is detected using a machine learningmodel that is trained to detect tables based on at least one of: tablemarkings associated with historical tables depicted in historical imagesof historical documents; text columns associated with the historicaltables; text rows associated with the historical tables; or tableidentifiers associated with the historical tables.
 3. The method ofclaim 2, wherein the historical documents are associated with thedocument based on the historical documents and the document involvingleast one of: a same type of document; a same type of table; or a sameindividual or organization.
 4. The method of claim 1, furthercomprising: prior to analyzing the table image, verifying that the tableof information includes a single table based on at least one of: headerinformation of the table of information; a table identifier of the tableof information; or table markings of the table of information.
 5. Themethod of claim 1, wherein the data extraction score is determined basedon at least one of: a rotational position of the table; a tilt angle ofthe table; or a amount of noise associated with the table.
 6. The methodof claim 1, wherein the morphological operation comprises at least oneof: a rotation of the table image; a reduction to a tilt of the table;or a reduction of an amount of noise associated with the table.
 7. Themethod of claim 1, wherein performing the action comprises at least oneof: providing, via a display of a user interface, the information;storing the information in a data structure in association with adigitized version of the document; integrating the information intorelated data associated with related documents that are associated withthe document; or retraining, based on feedback associated with theinformation, a machine learning model associated with identifying thetable and performing the morphological operation.
 8. A device,comprising: one or more processors configured to: identify, using atable detection model, a table of information that is depicted in animage of a document; extract, from the image of the document, a tableimage that depicts the table of information; determine, based on a dataextraction score associated with the table image, that the table ofinformation is to be enhanced; perform, based on the data extractionscore indicating that the table image is to be enhanced, a morphologicaloperation to the table image to generate an enhanced table image thatcorresponds to an enhanced table of information associated with thetable of information; process, using a data conversion technique, theenhanced table image to extract the information from the table; andstore the information as digitized table data.
 9. The device of claim 8,wherein the table detection model comprises a machine learning modelthat is trained to detect tables based on historical images ofhistorical documents that are associated with the document andhistorical configurations of tables that are depicted in the historicalimages.
 10. The device of claim 9, wherein the machine learning modelcomprises at least one of: a neural network trained to identifyconfigurations of table markings; a neural network trained to identifyarrangements of text; a neural network trained to identify tableidentifiers; or a neural network trained to identify types of tablesassociated with the document.
 11. The device of claim 8, wherein thedata extraction score is representative of a probability that the dataconversion technique can be used to convert the table image to digitizedtable data.
 12. The device of claim 8, wherein the one or moreprocessors are further configured to, when performing the morphologicaloperation: determine, based on an analysis of the table image, an issueassociated with using the data conversion technique to extract theinformation from the table; select, from a plurality of morphologicaloperations, the morphological operation that is associated with theissue; and apply the morphological operation to the table image.
 13. Thedevice of claim 12, wherein the issue comprises at least one of: thetable being rotated in the table image relative to other text of thedocument; the table being tilted in the table image; table markings ofthe table being unclear; or text of the table being unclear.
 14. Thedevice of claim 8, wherein the one or more processors are furtherconfigured to, when storing the extracted information as digitized tabledata, at least one of: identify related data associated with theextracted information, wherein the related data is associated withrelated documents that are associated with the document; and integratethe information with the related data associated with the relateddocuments that are associated with the document.
 15. A non-transitorycomputer-readable medium storing a set of instructions, the set ofinstructions comprising: one or more instructions that, when executed byone or more processors of a device, cause the device to: analyze a tableimage that is associated with a table of information, to extract tableimage data from the table image; determine a data extraction scoreassociated with the table image data, wherein the data extraction scoreis associated with whether a data conversion technique can be used toconvert the table image data to digitized table data; perform, based onthe data extraction score not satisfying a threshold, a morphologicaloperation to the table image data to generate an enhanced table imagethat corresponds to an enhanced table of information associated with thetable of information; process, using the data conversion technique, theenhanced table image to extract the information from the enhanced tableimage; and perform an action associated with the extracted information.16. The non-transitory computer-readable medium of claim 15, wherein theone or more instructions, when executed by the one or more processors,further cause the device to: prior to analyzing the table image, extractthe table image from an image of a document that depicts the table ofinformation, wherein the table of information was identified using amachine learning model that is trained to detect tables.
 17. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, when executed by the one or more processors, furthercause the device to: prior to performing the morphological operation,determine that the data extraction score is associated with an issuethat prevents the data conversion technique from being able to convertthe table image to digitized table data; and select, from a plurality ofmorphological operations, the morphological operation based on theissue.
 18. The non-transitory computer-readable medium of claim 17,wherein the issue comprises at least one of: the table being rotated inthe table image relative to other text in a document that includes thetable; the table being tilted in the table image; table markings of thetable being unclear; or text of the table being unclear.
 19. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, that cause the device to process the enhanced tableimage, cause the device to: perform a position-based extraction on theenhanced table image to identify fields of the table in the enhancedtable image based on table markings of the enhanced table image; orperform text-based extraction on the enhanced table to identify textwithin the fields of the table in the enhanced table image.
 20. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, that cause the device to perform the action, causethe device to: provide, via a display of a user interface, theinformation; store the information in a data structure in associationwith a digitized version of a document that includes the table;integrate the information into related data associated with relateddocuments that are associated with the document; or retrain, based onfeedback associated with the information, a machine learning modelassociated with identifying the table and performing the morphologicaloperation.